An Object-Oriented Approach to Nested Data Parallelism

This paper describes an implementation technique for integrating nested data parallelism into an object-oriented language. Data-parallel programming employs sets of data called “collections” and expresses parallelism as operations performed over the elements of a collection. When the elements of a collection are also collections, then there is the possibility for “nested data parallelism.” Few current programming languages support nested data parallelism however. In an object-oriented framework, a collection is a single object. Its type defines the parallel operations that may be applied to it. Our goal is to design and build an object-oriented data-parallel programming environment supporting nested data parallelism. Our initial approach is built upon three fundamental additions to C++. We add new parallel base types by implementing them as classes, and add a new parallel collection type called a “vector” that is implemented as a template. Only one new language feature is introduced: the foreach construct, which is the basis for exploiting elementwise parallelism over collections. The strength of the method lies in the compilation strategy, which translates nested data-parallel C++ into ordinary C++. Extracting the potential parallelism in nested foreach constructs is called “flattening” nested parallelism. We show how to flatten foreach constructs using a simple program transformation. Our prototype system produces vector code which has been successfully run on workstations, a CM-2 and a CM-5.